A Three-Tiered Evaluation Approach for Interactive Spoken Dialogue Systems
نویسندگان
چکیده
We describe a three-tiered approach for evaluation of spoken dialogue systems. The three tiers measure user satisfaction, system support of mission success and component performance. We describe our use of this approach in numerous fielded user studies conducted with the U.S. military.
منابع مشابه
From single word to natural dialogue
Spoken language dialogue systems represent the peak of achievement in speech technologies in the 20 th century and appear set to form the basis for the increasingly natural interactive systems to follow in the coming decades. This chapter first presents a model of the task-oriented spoken dialogue system, its multiple aspects and some of the remaining research challenges. In the context of this...
متن کاملEvaluating Interactive Dialogue Systems: Extending Component Evaluation to Integrated System Evaluation
This paper discusses the range of ways in which spoken dialogue system components have been evaluated and discusses approaches to evaluation that attempt to integrate component evaluation into an overall view of system performance. We will argue that the PARADISE (PARAdigm for Dialogue System Evaluation) framework has several advantages over other proposals.
متن کاملAdaptive Spoken Dialogue Systems
Adaptive systems cover a broad range of interactive systems which adjust to new tasks, situations, users or expressions. These systems identify and classify relevant features to develop over time, adjusting their behaviour to different users and situations. The topic of this paper is adaptation in spoken dialogue systems based on features in the dialogue. These systems automatically extract dia...
متن کاملDevelopment and Evaluation of the Spoken Dialogue System Based on the W3C Recommendations
Due to progress in technology of speech recognition and understanding, the Spoken dialogue systems (SDS) have started to emerge as a practical alternative for a conversational computer interface. They are more effective than Interactive Voice Response (IVR) systems since they allow a more free and natural interaction. The Spoken dialogue systems are designed for providing automatic dialogue-bas...
متن کاملRadiobot-CFF: a spoken dialogue system for military training
We describe a spoken dialogue system which can engage in Call For Fire (CFF) radio dialogues to help train soldiers in proper procedures for requesting artillery fire missions. We describe the domain, an information-state dialogue manager with a novel system of interactive information components, and provide evaluation results.
متن کامل